,

p

,

෍|ݓ௜|

ோ

௜ୀ଴

൑ݏ



d on this constraint, the objective function of Lasso is shown

here ߣ0 is a constraint constant and d stands for the number of

ent variables,

ܱൌ|܆^௧ܟെܡ|^ଶ൅ߣ෍|ݓ௜|

ௗ

௜ୀ଴





L1 constraint effect is more stringent. For instance, suppose there

variables, thus there are two regression coefficients, i.e., ݓଵ and

use |ݓଵ| ൅|ݓଶ| ൑ݏ, if |ݓଵ| increases, |ݓଶ| decreases. Suppose

and ݓଶ are positive. If ݓଵ is increased by ߜ, ݓଶ will be decreased

ݓଶൌݏെݓଵ. The relationship between the old value of ݓଶ and

value of ݓଶ is shown below,

ݓଶ

௡௘௪൑ݏെሺݓଵ൅ߜሻ൏ݏെݓଵൌݓଶ

௢௟ௗ

(4.43)

use of the use of L1 constraint, when ݓଵ increases, the decreasing

ݓଶ in a Lasso model is faster than that in a RLR model. This can

sed using Figure 4.20(a). Suppose ݓଶ is fixed and suppose two

he models have the same constraint constant C. The values of ݓଵ

RLR model and the Lasso model will have different strength,

e shown below,

ݓଵ

ோ௜ௗ௚௘ൌඥݏെݓଶൌ√ܥ

ݓଵ

௅௔௦௦௢ൌݏെݓଶൌܥ

(4.44)

hus no doubt that the following inequality is valid,

ݓଵ

ோ௜ௗ௚௘ൌ√ܥ൏ܥൌݓଵ

௅௔௦௦௢

(4.45)